Color modification is a class of operations that may be performed an image (e.g., a digital image) to change colors of the image. Such color modifications may be made to correct color errors due to process errors and to adjust the colors used in the video for artistic expression. Such color modifications may include enhancing contrasts or color in an image to give a program an overall xe2x80x9clook,xe2x80x9d or applying special effects to selected segments. Other color modifications may be made by a user (e.g., an editor or colorist) to correct problems with color or lighting resulting from the source of the media. Such corrections may include color balancing for camera and lighting differences, correcting for film processing differences, matching colors and tones from shot to shot, or adjusting video levels for differences in source tapes, source decks, etc.
Such color modifications may be made on a computer or other digital-based system such as, for example, a digital non-linear editing (DNLE) system. DNLE is a process by which digital media may be edited. DNLE, as the name implies, is performed on digital media stored as data in digital media files on a digital random access medium. DNLE may be conducted in a non-linear fashion because the digital media files in which the digital media is stored can be randomly accessed. Thus a user may access a piece of the digital media without having to proceed sequentially through other pieces of the digital media stored in the same or other digital media files. More than one user also may be able to access different pieces of the same digital media contemporaneously. The digital media may be a digitized version of a film or videotape or digital media produced through live capture onto a disk of a graphics or animation software application. Example commercial DNLE systems include the Media Composer(copyright) or Symphony video production systems or NewsCutter(copyright) news editing system available from Avid Technology, Inc. For a more detailed description of DNLE, see Digital Nonlinear Editing, New Approaches to Editing Film and Video, 1993, by Thomas Ohanian (hereinafter Ohanian). For a more detailed description of computer graphics in general, see Computer Graphics Principles and Practice, second edition, 1997, by James D. Foley et al. (hereinafter Foley).
Digital images are comprised of an array of picture elements called pixels. For a given image, color modifications may be applied to all pixels in the image or pixels comprising a portion of the image. In digital video signal processing, a variety of data formats can be used to represent the color of pixels within a digital image. Formats may be classified into two major categories: composite signals and component signals. Component formats represent a color as multiple components, each component defining a value along a dimension of the color space in which the color being represented is defined. A composite video signal is an analog signal that uses a high frequency subcarrier to encode color information. The subcarrier is a sinewave of which the amplitude is modulated by the saturation of the color represented by the signal, and the hue of the color is encoded as a phase difference from a color burst. Analog composite signals are generally used to broadcast television video signals.
There are a variety of color spaces corresponding to component formats that may be used to represent color such as, for example, RGB, HSL, and YCbCr. RGB (Red, Green, Blue) color space represents a color with a red component, a green component and a blue component. In a three-dimensional coordinate system, each component of the RGB color space represents a value along an axis, the combination of the values defining a cubic color space.
The HSL (Hue, Saturation, Lightness or Luminance) color space represents a color with a hue component, H, a saturation component, S, and a luma component, L. In a three-dimensional HSL coordinate system, the luma component, Y, represents a value along a luma axis, the hue component, H, represents the angle of a chroma region with respect to the luma axis and the saturation component, S, represents the magnitude of the chroma region. The combination of the values defines a hexagonal cone-shaped color space.
The YCbCr color space represents a color with a luma component Y, and two chroma components, Cr and Cb. In a three-dimensional YCbCr coordinate system, each component of the YCbCr color space represents a value along an axis, the combination of the values defining a cylindrical color space around the luma axis. The chroma components, Cr and Cb, define the chroma region.
In either HSL or YCbCr color space, the luma component, L or Y, respectively, may be used independently to represent a pixel in a black and white (i.e. grayscale) image to be displayed, for example, with a black and white monitor. A pixel color represented with only a luma value may be referred to herein as having a grayscale value.
Further, in both HSL and YCbCr color space, the components defining a chroma (i.e., chrominance) chroma region, HS or CbCr, respectively, may be referred to herein as chroma components and may be referred to as defining a chroma. A plane defined by all possible values of chroma components for a given value of a luma may be referred to herein as a chroma plane. A two-dimensional region defined for such a chroma region may be referred to herein as a two-dimensional chroma region or a chroma region. Such a chroma region may be orthogonal to a luma axis. The term xe2x80x9cchromaxe2x80x9d may be used herein interchangeably with the term xe2x80x9cchrominance.xe2x80x9d
Color modifications may be specified to affect all pixels of a digital image or less than all pixels of a digital image. A color modification defined to affect less than all pixels of a digital image may be referred to herein as a xe2x80x9csecondary color modificationxe2x80x9d. A user may specify a secondary color modification by specifying that color modification be applied only to pixels that meet certain criteria such as, for example, positional (e.g. coordinate-based) and color-space-based (i.e. component-based) criteria. For color space-based criteria, a user may specify, for example, that a secondary color modification be applied to all pixels of a digital image that have a chroma component that has a value within a certain range. In an HSL color space, for example, a user may specify that the secondary color modification be applied to all colors that fall within a range of hue and saturation values.
Various color modifications and support for managing them are described in U.S. patent application Ser. No. 09/392,823, entitled xe2x80x9cModification of Media with Common Attributes on a Digital Nonlinear Editing Systemxe2x80x9d (the Gonsalves I application) by Robert Gonsalves and Michael D. Laird filed Sep. 9, 1999, and in U.S. patent application Ser. No. 09/293,732, entitled xe2x80x9cMulti-tone Representation of a Digital Image on a Digital Nonlinear Editing Systemxe2x80x9d (the Gonsalves II application), by Robert Gonsalves, filed Apr. 16, 1999. The contents of both these applications are herein incorporated by reference. Suitable commercial systems for color modification include the Media Composer(copyright) and Symphony video production systems, and Avid Media Illusion(trademark), all available from Avid Technology, Inc. The Avid Media Illusion Reference Guide, available from Avid Technology, Inc. is herein incorporated by reference. Other commercial software applications may be used, including Adobe Photoshop from Adobe Systems Inc., Flame(copyright) from Discreet Logic, Inc, a division of Autodesk, Inc., and Renaissance 8:8:8 from daVinci Systems, Inc.
Some color modification systems that provide secondary color modification allow a user to define a chroma region for which to apply a secondary color modification. To define a chroma region, such systems typically provide a standard shape such as, for example, a rectangle or wedge-like shape. A rectangle may define a chroma region using Euclidean coordinates, such as for example, Cr and Cb, where both Cr and Cb represent a displacement from a luma axis on a chroma plane. A rectangle may be defined as a chroma region bound by a minimum and maximum Cr value and a minimum and maximum Cb value.
A wedge-like shape may define a chroma region using polar coordinates, such as, for example, H and S, where H represents an angular displacement between a radial line and a reference radial line, both lines extending radially from the luma axis on a chroma plane, and S represents a magnitude of such a radial line. A wedge shape may be defined as a chroma region bound by a minimum and maximum saturation and a minimum and maximum hue.
A problem with typical color modification systems that provide secondary chroma modifications is that the standard shapes that a user may define are sometimes not ideal for performing secondary chroma modification on discrete spatial regions of a digital image such as, for example, a section of sky or a person""s skin. Specifically, the standard shapes may not provide a user the capability to specify a chroma region that closely correlates to a range of chroma values for the pixels of a discrete spatial region. Thus, the defined shape may capture chroma not intended by a user to be chroma modified, and may not capture other chroma that the user intended to chroma modify.
The range of many discrete spatial regions of a digital image, when mapped to a chroma plane, form approximately an ellipsoid oriented along a radial line extending from the luma axis of a chroma region. For the reasons described above, standard chroma region shapes such as, for example, rectangles and wedge-shapes may be ill-suited to represent such ellipsoids. Further, a typical wedge-like chroma region is defined to include, for each value of hue within such chroma region, the maximum value of saturation allowed for the chroma plane. Such a wedge-shaped region may be ideal for highly saturated images (i.e. an image having relatively many pixels that have a high saturation value such as, for example, a bright yellow frisbee on a sunny day); however, many digital images and, consequently, many discrete spatial regions of a typical digital image, are lowly saturated. Consequently, a wedge-like chroma region may capture several undesired chroma.
Another problem with typical color modification systems that provide secondary chroma modifications is that such a system does not provide a convenient means for defining an manipulating a chroma region that closely correlates to a discrete region of an actual digital image. Typically, a user must define the chroma region, and then apply it to a digital image to determine if the defined chroma region captures all or enough of the pixels of the discrete spatial region. If the desired pixels are not captured, the user may repeatedly redefine the chroma region until the chroma region captures the desired pixels.
Accordingly, in an aspect, an ellipsoid chroma region of a chroma plane may be defined to specify a range of chromas for which a secondary chroma modification is to be applied, and an output chroma for each chroma of the chroma plane may be determined based on the ellipsoid chroma region.
In another aspect, two or more different-shaped chroma regions of a chroma plane may be defined and manipulated, each chroma region specifying a range of chroma for which a secondary chroma modification is to be applied, and an output chroma for each chroma of a chroma plane may be determined based on the two or more different-shaped chroma regions.
In yet another aspect, chroma-matching techniques may be used to define and manipulate a chroma region of a chroma plane that specifies a range of chroma for which a secondary chroma modification is to be applied, and an output chroma for each chroma of the chroma plane may be determined based on the chroma region.
The several embodiments of a system and method of performing secondary color modification described herein may be implemented using software, hardware, firmware, or any combination thereof, and may be embodied in a computer program product.